Image processing apparatus and image processing method

ABSTRACT

An image processing apparatus includes: an image processing apparatus comprising: a template image extracting section that extracts a template image from blade images obtained by capturing blades periodically arrayed in a jet engine; an image comparing section that compares the template image with the blade images; an image selecting section that selects an image from the blade images based on a result of the image comparison of the image comparing section; and a first difference extracting section that extracts a difference between the template image and the image selected by the image selecting section.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus and animage processing method.

Priority is claimed on Japanese Patent Application Nos. 2009-168012filed on Jul. 16, 2009 and 2009-179323 filed on Jul. 31, 2009, thecontent of which is incorporated herein by reference.

2. Description of Related Art

Conventionally, in order to inspect blades in a jet engine, the bladesare observed using an observation jig, such as an endoscope. Forexample, a method of detecting defects in blades by imaging the bladessequentially and comparing two sequential images with each other isdisclosed in U.S. Patent Application Publication No. 2004/183900. Inaddition, a method of detecting defects in blades on the basis of thefeature amount of known defect pattern is disclosed in JapaneseUnexamined Patent Application, First Publication No. 2007-163723.

SUMMARY OF THE INVENTION

According to an aspect of the invention, there is provided an imageprocessing apparatus which includes: a template image extracting sectionthat extracts a template image from blade images obtained by capturingblades periodically arrayed in a jet engine; an image comparing sectionthat compares the template image with the blade images; an imageselecting section that selects an image from the blade images based on aresult of the image comparison of the image comparing section; and afirst difference extracting section that extracts a difference betweenthe template image and the image selected by the image selectingsection.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the configuration of a bladeinspection system according to a first embodiment of the invention;

FIG. 2 is a block diagram showing the configuration of an endoscopeapparatus included in the blade inspection system according to the firstembodiment of the invention;

FIG. 3 is a block diagram showing the configuration of a bladeinspection system (modification) according to the first embodiment ofthe invention;

FIG. 4 is a block diagram showing the configuration of a bladeinspection system (modification) according to the first embodiment ofthe invention;

FIG. 5 is a block diagram showing the configuration of a PC included inthe blade inspection system (modification) according to the firstembodiment of the invention;

FIG. 6 is a reference view showing a screen of blade recording softwareaccording to the first embodiment of the invention;

FIG. 7 is a reference view showing a screen of blade recording softwareaccording to the first embodiment of the invention;

FIG. 8 is a reference view showing the directory structure in a memorycard according to the first embodiment of the invention;

FIG. 9 is a reference view showing a save folder list according to thefirst embodiment of the invention;

FIG. 10 is a reference view showing an image file list according to thefirst embodiment of the invention;

FIG. 11 is a flow chart showing the procedure of an operation based onblade recording software according to the first embodiment of theinvention;

FIG. 12 is a flow chart showing the procedure of an operation based onblade recording software according to the first embodiment of theinvention;

FIG. 13 is a flow chart showing the procedure of an operation based onblade recording software according to the first embodiment of theinvention;

FIG. 14 is a flow chart showing the procedure of an operation based onblade recording software according to the first embodiment of theinvention;

FIG. 15 is a flow chart showing the procedure of an operation based onblade recording software according to the first embodiment of theinvention;

FIG. 16 is a flow chart showing the procedure of an operation based onblade recording software according to the first embodiment of theinvention;

FIG. 17 is a graph showing a temporal change of the correlation valueaccording to the first embodiment of the invention;

FIG. 18 is a flow chart showing the procedure of an operation based onblade recording software according to the first embodiment of theinvention;

FIG. 19 is a flow chart showing the procedure of an operation based onblade recording software according to the first embodiment of theinvention;

FIG. 20 is a flow chart showing the procedure of an operation based onblade recording software according to the first embodiment of theinvention;

FIG. 21 is a flow chart showing the procedure of an operation based onblade recording software according to the first embodiment of theinvention;

FIG. 22 is a reference view showing a screen of blade recording softwareaccording to a second embodiment of the invention;

FIG. 23 is a reference view showing a screen of blade recording softwareaccording to the second embodiment of the invention;

FIG. 24 is a reference view showing a screen of blade recording softwareaccording to the second embodiment of the invention;

FIG. 25 is a reference view showing a screen of blade recording softwareaccording to the second embodiment of the invention;

FIG. 26 is a reference view showing a screen of blade recording softwareaccording to the second embodiment of the invention;

FIG. 27 is a flow chart showing the procedure of an operation based onblade recording software according to the second embodiment of theinvention;

FIG. 28 is a flow chart showing the procedure of an operation based onblade recording software according to the second embodiment of theinvention;

FIG. 29 is a flow chart showing the procedure of an operation based onblade recording software according to the second embodiment of theinvention;

FIG. 30 is a flow chart showing the procedure of an operation based onblade recording software according to the second embodiment of theinvention;

FIG. 31 is a flow chart showing the procedure of an operation based onblade recording software according to the second embodiment of theinvention;

FIG. 32 is a flow chart showing the procedure of an operation based onblade recording software according to the second embodiment of theinvention;

FIG. 33 is a reference view for showing defect extraction processingaccording to the second embodiment of the invention;

FIG. 34 is a flow chart showing the procedure of an operation based onblade recording software according to the second embodiment of theinvention;

FIG. 35 is a flow chart showing the procedure of an operation based onblade recording software according to the second embodiment of theinvention;

FIGS. 36A to 36F are reference views for explaining defect designationprocessing according to the second embodiment of the invention;

FIG. 37 is a reference view showing a screen of blade recording softwareaccording to the second embodiment of the invention;

FIG. 38 is a reference view showing a screen of blade recording softwareaccording to the second embodiment of the invention;

FIG. 39 is a flow chart showing the procedure of an operation based onblade recording software according to a third embodiment of theinvention;

FIG. 40 is a flow chart showing the procedure of an operation based onblade recording software according to the third embodiment of theinvention;

FIG. 41 is a flow chart showing the procedure of an operation based onblade recording software according to the third embodiment of theinvention;

FIG. 42 is a flow chart showing the procedure of an operation based onblade recording software according to the third embodiment of theinvention;

FIG. 43 is a flow chart showing the procedure of an operation based onblade recording software according to the third embodiment of theinvention;

FIG. 44 is a reference view showing a screen of blade recording softwareaccording to the third embodiment of the invention;

FIG. 45 is a reference view showing a screen of blade recording softwareaccording to the third embodiment of the invention;

FIG. 46 is a reference view showing a screen of blade recording softwareaccording to the third embodiment of the invention;

FIG. 47 is a flow chart showing the procedure of an operation based onblade recording software according to the third embodiment of theinvention;

FIG. 48 is a flow chart showing the procedure of an operation based onblade recording software according to the third embodiment of theinvention;

FIG. 49 is a flow chart showing the procedure of an operation based onblade recording software according to the third embodiment of theinvention;

FIG. 50 is a flow chart showing the procedure of an operation based onblade recording software according to the third embodiment of theinvention;

FIG. 51 is a flow chart showing the procedure of an operation based onblade recording software according to the third embodiment of theinvention;

FIG. 52 is a flow chart showing the procedure of an operation based onblade recording software according to the third embodiment of theinvention;

FIG. 53 is a reference view showing blade region extraction processingaccording to the third embodiment of the invention;

FIG. 54 is a reference view showing blade region extraction processingaccording to the third embodiment of the invention;

FIG. 55 is a graph showing the average luminance of blade regionsaccording to the third embodiment of the invention;

FIG. 56 is a flow chart showing the procedure of an operation based onblade recording software according to the third embodiment of theinvention; and

FIG. 57 is a flow chart showing the procedure of an operation based onblade recording software according to the third embodiment of theinvention.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, embodiments of the invention will be described withreference to the accompanying drawings.

First Embodiment

First, a first embodiment of the invention will be described. FIG. 1shows the configuration of a blade inspection system according to thepresent embodiment. In a jet engine 1, a plurality of turbine blades 10(or compressor blade) to be inspected are periodically arrayed atpredetermined intervals. In addition, a turning tool 2 which rotates theturbine blades 10 in a rotation direction A at the predetermined speedis connected to the jet engine 1. In the present embodiment, the turbineblades 10 are always rotated while an image of the turbine blades 10 isbeing captured.

In the present embodiment, an endoscope apparatus 3 (corresponding to animage processing apparatus of the invention) is used to acquire theimage of the turbine blades 10. An endoscope insertion section 20 of theendoscope apparatus 3 is inserted into the jet engine 1, so that animage of the rotating turbine blades 10 is exported through theendoscope insertion section 20. In addition, blade recording softwarefor recording an image, which is obtained by imaging the turbine blades10 at the desired angle, is stored in the endoscope apparatus 3.

FIG. 2 shows the configuration of the endoscope apparatus 3. Theendoscope apparatus 3 is configured to include the endoscope insertionsection 20, an endoscope apparatus body 21, a monitor 22, and a remocon(remote controller) 23. An imaging optical system 30 a and an imagingelement 30 b are provided at the tip of the endoscope insertion section20. In addition, an image signal processor (CCU) 31, a light source 32,a curve control unit 33, and a computer 34 for control are provided inthe endoscope apparatus body 21.

In the endoscope insertion section 20, the imaging optical system 30 acondenses light from a subject and forms the subject image on theimaging surface of the imaging element 30 b. The imaging element 30 bgenerates an imaging signal by performing photoelectric conversion ofthe subject image. The imaging signal output from the imaging element 30b is input to the image signal processor 31.

In the endoscope apparatus body 21, the image signal processor 31converts the imaging signal from the imaging element 30 b into a videosignal, such as an NTSC signal and supplies the video signal to thecomputer 34 for control. If necessary, the image signal processor 31outputs the video signal to the outside as analog video output.

The light source 32 is connected to the tip of the endoscope insertionsection 20 through an optical fiber or the like, so that the lightsource 32 can irradiate the light to the outside. The curve control unit33 is connected to the tip of the endoscope insertion section 20, sothat the curve control unit 33 can curve the tip vertically andhorizontally. Control of the light source 32 and the curve control unit33 is performed by the computer 34 for control.

The computer 34 for control is configured to include a RAM 34 a, a ROM34 b, a CPU 34 c, a network I/F 34 d, an RS232C I/F 34 e, and a card IF34 f as external interfaces. The RAM 34 a is used to temporarily storethe data, such as image information required for software operation. Aseries of software for controlling the endoscope apparatus 3 is storedin the ROM 34 b. Blade recording software, which will be describedlater, is also stored in the ROM 34 b. The CPU 34 c executes operationsand the like for various kinds of control using the data stored in theRAM 34 a according to a command code of software stored in the ROM 34 b.

The network I/F 34 d is an interface for connection with an external PCusing a LAN cable. Through the network I/F 34 d, the image informationoutput from the image signal processor 31 can be transmitted to theexternal PC. The RS232C I/F 34 e is an interface for connection with theremocon 23. A user can control various operations of the endoscopeapparatus 3 by operating the remocon 23. Various memory cards 50, whichare recording media, may be freely mounted in the card I/F 34 f orreleased from the card I/F 34 f. If the memory card 50 is mounted, datasuch as the image information stored in the memory card 50 may beexported or the data such as the image information may be recorded inthe memory card 50 by control of the CPU 34 c.

As a modification of the configuration of the blade inspection system inthe present embodiment, the configuration shown in FIG. 3 may be used.In this modification, a video terminal cable 4 and a video capture card5 are connected to the endoscope apparatus 3. By the video terminalcable 4 and the video capture card 5, an image captured by the endoscopeapparatus 3 may be exported to a PC 6 (corresponding to the imageprocessing apparatus of the invention). The PC 6 is shown as a notebookPC in FIG. 3, but it may be a desktop PC or the like. In addition, bladerecording software for recording an image, which is obtained by imagingthe turbine blades 10 at the desired angle, is stored in the PC 6.

In addition, although the video terminal cable 4 and the video capturecard 5 are used to export an image to the PC 6 in FIG. 3, a LAN cable 7may be used as shown in FIG. 4. The endoscope apparatus 3 includes anetwork I/F 34 d through which the captured image can be loaded on a LANnetwork. In addition, an image can be exported to the PC 6 through theLAN cable 7.

FIG. 5 shows the configuration of the PC 6. The PC 6 includes a PC body24 and a monitor 25. A computer 35 for control is provided in the PCbody 24. The computer 35 for control is configured to include a RAM 35a, an HDD (hard disk drive) 35 b, a CPU 35 c, a network I/F 35 d, and aUSB I/F 35 e as external interfaces. The computer 35 for control isconnected to the monitor 25 so that the image information, a softwarescreen, and the like are displayed on the monitor 25.

The RAM 35 a is used to temporarily store the data, such as imageinformation required for software operation. A series of software forcontrolling the endoscope apparatus is stored in the HDD 35 b. Bladerecording software is also stored in the HDD 35 b, Moreover, in thepresent embodiment, a save folder for saving an image of the turbineblade 10 is set in the HDD 35 b. The CPU 35 c executes operations andthe like for various kinds of control using the data stored in the RAM35 a according to a command code of software stored in the HDD 35 b.

The network I/F 35 d is an interface for connecting the endoscopeapparatus 3 with the PC 6 using the LAN cable 7. Through the network I/F35 d, the image information out put through the LAN cable 7 from theendoscope apparatus 3 can be input to the PC 6. The USB I/F 35 e is aninterface for connecting the endoscope apparatus 3 with the PC 6 usingthe video capture card 5. Through the USB I/F 35 e, the imageinformation output as analog video from the endoscope apparatus 3 can beinput to the PC 6.

In the blade inspection system shown in FIGS. 3 and 4, the same effectsas in the blade inspection system shown in FIG. 1 can be obtained.Especially in the case where the performance of the endoscope apparatusis inferior to the performance of the PC and the operation speed or thelike of the endoscope apparatus is not enough, the blade inspectionsystem shown in FIGS. 3 and 4 is effective.

Next, a screen of blade recording software will be described. FIG. 6shows a main window of the blade recording software. A main window 600shown in FIG. 6 is displayed when a user starts the blade recordingsoftware.

The main window 600 is displayed according to the control of the CPU 34c. The CPU 34 c generates a graphic image signal (display signal) fordisplaying the main window 600 and outputs the graphic image signal tothe monitor 22. Moreover, when displaying images captured by theendoscope apparatus 3 (hereinafter, described as endoscope images) onthe main window 600 so that they superimpose each other, the CPU 34 cperforms a processing of superimposing the image data exported from theimage signal processor 31 on the graphic image signal and outputs asignal after the processing (display signal) to the monitor 22.

In addition, when updating a display state of a GUI on the main window600, the CPU 34 c generates a graphic image signal corresponding to themain window 600 after updating and performs the same processing asabove. Processing related to the display of windows other than the mainwindow 600 is the same as above. Hereinafter, processing when the CPU 34c generates a graphic image signal in order to display the main window600 and the like (including update) is described as processing fordisplaying the main window 600 and the like.

The user can view an endoscope image and save an image file by operatingthe main window 600 through the remocon 23 using a GUI (graphical userinterface) function. Hereinafter, functions of various GUIs will bedescribed.

A [preview image] box 601, a [template image] box 602, and a [recordimage] box 603 are disposed in the upper part of the main window 600.

The [preview image] box 601 is a box for displaying an endoscope image.If a [preview start] button 610, which will be described later, ispressed when the turbine blades 10 are rotating by the turning tool 2,an endoscope image (image showing that the turbine blades 10 arerotating) is displayed in real time. Thus, the user can view theendoscope image through the [preview image] box 601. Hereinafter,displaying an endoscope image in the [preview image] box 601 isdescribed as a preview.

The [template image] box 602 is a box for displaying a template image.If a [template registration] button 612 to be described later ispressed, an image of one frame captured at that timing among images ofrespective frames, which form the endoscope image, is displayed as atemplate image in the [template image] box 602. The template image is animage as a reference when displaying a record image, which will bedescribed later.

The [record image] box 603 is a box for displaying a record image to bedescribed later. After a [record start] button 613 to be described lateris pressed, images (hereinafter, described as record images), which arehighly correlated with the template image, among images of respectiveframes that form the endoscope image are sequentially displayed. Therecord images displayed in the [record image] box 603 are sequentiallysaved as image files in a save folder in the memory card 50.

Hereinafter, the image file saved here is described as a record imagefile. In addition, saving the record image files in the save folder inthe memory card 50 is described as a record hereinbelow. Details of thesave folder will be described later.

The [preview start] button 610 is a button for starting the display ofan endoscope image in the [preview image] box 601. A [preview stop]button 611 is a button for stopping the display of an endoscope image inthe [preview image] box 601.

The [template registration] button 612 is a button for registering adesired image as a template image. If the [template registration] button612 is pressed, an image of one frame captured at that timing amongimages of respective frames, which form the endoscope image, isdisplayed as a template image in the [template image] box 602. Inaddition, the image of one frame is recorded as a template image in theRAM 34 a. An operation until the image of one frame is recorded as atemplate image in the RAM 34 a after the [template registration] button612 is pressed is performed by a template extracting section 34 c ₁ ofthe CPU 34 c.

The [record start] button 613 is a button for starting record. If the[record start] button 613 is pressed, the value of a [record number] box620, which will be described later, is reset to 0. Then, the endoscopeimage and the template image are compared with each other for everyframe and the record image of one frame, which are highly correlatedwith the template image, among the images of respective frames whichform the endoscope image are displayed sequentially in the [recordimage] box 603. Moreover, the displayed record images are sequentiallysaved as image files in a save folder in the memory card 50. Thecomparison between the endoscope image and the template image isperformed for every frame and the record image of one frame, which arehighly correlated with the template image, are displayed sequentially inthe [record image] box 603. Here, among the operations in which thedisplayed record images are being saved sequentially as image files inthe save folder in the memory card 50, an operation of comparing anendoscope image with a template image is performed by an image comparingsection 34 c ₂ of the CPU 34 c and the subsequent operation is performedby an image selecting section 34 c ₃.

More specifically, the image of one frame when the position or angle ofthe turbine blade 10 in the endoscope image becomes equal to that of theturbine blade 10 in the template image (simply speaking, when theturbine blade 10 in the endoscope image and the turbine blade 10 in thetemplate image are viewed in the same way) are displayed and saved.

A [record stop] button 614 is a button for stopping record. An [imagebrowse] button 615 is a button for browsing an image file saved in thesave folder in the memory card 50. If the [image browse] button 615 ispressed, an [image browse] window, which will be described later, isdisplayed. While the [image browse] window is being displayed, a useroperation on the main window 600 is invalid.

The [record number] box 620 is a box for displaying the number of recordimage files which are currently saved (hereinafter, described as arecord number). However, image files of the template image are notcounted. Moreover, as described above, the value of the [record number]box 620 is reset to 0 if the [record start] button 613 is pressed.

A [maximum record number] box 621 is a box for displaying the maximumnumber of record image files (hereinafter, described as a maximum recordnumber). If the record number becomes equal to the maximum record numberduring record, the record ends automatically. An arbitrary maximumrecord number may be input in the [maximum record number] box 621. Forexample, a required number of image files of the turbine blades 10 canbe saved by inputting the number of blades corresponding to one round ofthe turbine blades 10 in the [maximum record number] box 621.

An [end] button 630 is a button for ending the blade recording software.If the [end] button 630 is pressed, the main window 600 is not displayedand the operation of the blade recording software ends.

FIG. 7 shows an [image browse] window of the blade recording software.An [image browse] window 700 shown in FIG. 7 is displayed when the[image browse] button 615 of the main window 600 is pressed as describedabove.

A user can browse a record image file by operating the [image browse]window 700 through the remocon 23 using a GUI function. Hereinafter,functions of various GUIs will be described.

A [browse image] box 701 is a box for displaying a record image file. Ifa [<<back] button 710 or a [next>>] button 711, which will be described,is pressed or if the selection of a [date and time selection] box 724 ischanged, a record image file displayed in the [browse image] box 701 ischanged. The user can browse a record image file through the [browseimage] box 701. Hereinafter, a record image displayed in the [browseimage] box 701 is described as a browse image, and the image filethereof is described as a browse image file.

The [<<back] button 710 is a button for changing an browse image. If the[<<back] button 710 is pressed, an image file with an image file No(image file number), which is smaller by 1 than the image file No of theimage file displayed in the [browse image] box 701, among an image filelist to be described later is displayed. Then, an image file namedisplayed in an [image file name] box 720, which will be describedlater, is also changed.

The [next>>] button 711 is also a button for changing a browse image. Ifthe [next>>] button 711 is pressed, an image file with an image file No,which is larger by 1 than the image file No of the image file displayedin the [browse image] box 701, among the image file list to be describedlater is displayed. Then, an image file name displayed in the [imagefile name] box 720, which will be described later, is also changed.

The [image file name] box 720 is a box for displaying a file name of abrowse image file. If the [<<back] button 710 or the [next>>] button 711is pressed or the selection of the [date and time selection] box 724 ischanged, display of the image file name of the [image file name] box 720is changed.

A [number of image files] box 721 is a box for displaying the number ofimage files in the image file list to be described later. If theselection of the [date and time selection] box 724 is changed, displayof the number of image files of the [number of image files] box 721 ischanged.

A [save date and time] box 722 is a box for displaying save date andtime of a browse image file. If the [<<back] button 710 or the [next>>]button 711 is pressed or the selection of the [date and time selection]box 724 is changed, the display of the save date and time of an imagefile of the [save date and time] box 722 is changed.

The [date and time selection] box 724 is a button for changing a browseimage. Record start date and time of a save folder list, which will bedescribed, is displayed in the list form in the [date and timeselection] box 724. If the selection of record start date and time ofthe [date and time selection] box 724 is changed, a record image filesaved in a save folder, which has the selected record start date andtime, is displayed in the [browse image] box 701. Then, display of theimage file name of the [image file name] box 720 and the number of theimage files of the [number of image files] box 721 are also changed.

A [close] button 730 is a button for ending the browse of the images. Ifthe [close] button 730 is pressed, the [image browse] window 701 is notdisplayed to return a state where the main window 600 is operated.

Next, a directory structure in the memory card 50 will be described withreference to FIGS. 8 to 10. As shown in FIG. 8, a directory locatedimmediately below the memory card 50 includes a plurality of savefolders 800. The save folder 800 is a folder in which a record imagefile is saved. The record start date and time becomes a folder name ofthe save folder 800. For example, if the record start date and time is“2007/12/26 21:32:21”, the folder name is set to “20071226_(—)213221”.

A directory located immediately below each save folder includes aplurality of record image files 810. The name of the record image filesare saved as “001.jpg”, “002.jpg”, “003.jpg”, in order in which therecord image files are saved. However, a file name of a template imagefile becomes “Temp.jpg”.

In addition, when an image browse processing to be described later isperformed, a save folder list and an image file list are created.

The save folder list is a list of save folders. As shown in FIG. 9, thesave folder list includes save folder No (save folder number), therecord start date and time, and a folder name. Numbers of 1, 2, 3, areassigned to the save folder No in the order in which save folders arecreated.

The image file list is a list of record image files saved in each savefolder. As shown in FIG. 10, the image file list includes an image fileNo, a file save date and time, and a file name. Numbers of 1, 2, 3, areassigned to the image file No in the order in which files are saved.However, a last image file No is assigned only to a template image.

Next, the flow of operation of blade recording software will bedescribed with reference to FIG. 11. In step SA, a user starts bladerecording software. In this case, on the basis of an instruction tostart the blade recording software which is input to the remocon 23, theCPU 34 c reads the blade recording software stored in the ROM 34 b intothe RAM 34 a and starts the operation according to the blade recordingsoftware. In step SB, the CPU 34 c performs a processing for displayingthe main window.

In step SC, the CPU 34 c performs initialization processing. Theinitialization processing is a processing of setting the initial statesof various GUIs within the main window or processing of setting theinitial values of various kinds of data recorded in the RAM 34 a.Details of the initialization processing will be described later. Instep SD, the CPU 34 c performs preview processing. The previewprocessing is a processing of starting and stopping the preview. Detailsof the preview processing will be described later.

In step SE, the CPU 34 c performs template registration processing. Thetemplate registration processing is a processing of displaying atemplate image in a [template image] box and recording the templateimage in the RAM 34 a. Details of the template registration processingwill be described later. In step SF, the CPU 34 c performs recordprocessing. The record processing is a processing of starting andstopping the record. Details of the record processing will be describedlater.

In step SG, the CPU 34 c performs the image browse processing. The imagebrowse processing is a processing that a user performs to browse arecord image file. Details of the image browse processing will bedescribed later. In step SH, processing branches according to whether ornot the user has pressed an [end] button. If the user has pressed the[end] button, the process proceeds to step SI. Moreover, if the user hasnot pressed the [end] button, the process proceeds to step SD. In stepSI, the CPU 34 c makes the main window be not displayed and ends theoperation of the blade recording software.

Next, the flow of initialization processing (step SC) will be describedwith reference to FIG. 12. In step SC1, the CPU 34 c invalidates alluser operations on the [preview stop] button, the [templateregistration] button, the [record start] button, and the [record stop]button. Hereinafter, it is simply described as “invalid” that a useroperation using a GUI, such as a button, is in an invalid state (forexample, gray state). Moreover, it is simply described as “valid” that auser operation using a GUI, such as a button, is in a valid state.

In step SC2, the CPU 34 c records, in the RAM 34 a, the record number Ras 0 and the maximum record number Rm as Ri. Ri is an initial value ofthe maximum record number Rm, and a predetermined value is recorded asRi in the RAM 34 a. In step SC3, the CPU 34 c performs a processing fordisplaying the record number R (=0) in the [record number] box. In stepSC4, the CPU 34 c performs a processing for displaying the maximumrecord number Rm in the [maximum record number] box.

In step SC5, the CPU 34 c sets all of a preview flag, a record flag, anda save flag to OFF and records them in the RAM 34 a. The preview flag isa flag indicating whether or not a current state is a preview state. Therecord flag is a flag indicating whether or not a current state is“under record”. The save flag is a flag indicating whether or not abuffer image, which will be described later, is saved as a record imagefile during the record. Hereinafter, all flags used during the operationof the blade recording software have values of ON or OFF. After theprocessing in step SC5 ends, the process proceeds to step SD.

Next, the flow of preview processing in step SD will be described withreference to FIG. 13. In step SD1, the CPU 34 c checks whether or notthe [preview start] button has been pressed by the user. If the [previewstart] button has been pressed, the process proceeds to step SD2. If the[preview start] button has not been pressed, the process proceeds tostep SD4.

In step SD2, the CPU 34 c makes the [preview start] button invalid, the[preview stop] button valid, and the [template registration] buttonvalid. In step SD3, the CPU 34 c sets a preview flag to ON and recordsit in the RAM 34 a.

In step SD4, the CPU 34 c checks whether or not the preview flagrecorded in the RAM 34 a is ON. If the preview flag is ON, the processproceeds to step SD5. If the preview flag is OFF, the process proceedsto step SD8.

In step SD5, the CPU 34 c acquires an image of one frame (image signal)from the image signal processor 31 as a frame image. In addition, at apoint of time before step SD5, the imaging element 30 b generates animaging signal of one frame, and the image signal processor 31 convertsthe imaging signal into a video signal to create an image of one frame.

In step SD6, the CPU 34 c records the frame image, which was acquired instep SD5, in the RAM 34 a. The frame image recorded in the RAM 34 a isoverwritten whenever the CPU 34 c acquires a frame image. In step SD7,the CPU 34 c performs a processing for displaying the frame imageacquired in step SD5 in the [preview image] box.

In step SD8, the CPU 34 c checks whether the [preview stop] button hasbeen pressed by the user. If the [preview stop] button has been pressed,the process proceeds to step SD9. If the [preview stop] button has notbeen pressed, the process proceeds to step SE.

In step SD9, the CPU 34 c makes the [preview start] button valid, the[preview stop] button invalid, and the [template registration] buttoninvalid. In step SD10, the CPU 34 c sets a preview flag to OFF andrecords it in the RAM 34 a. After the processing in step SD10 ends, theprocess proceeds to step SE.

Next, the flow of template registration processing in step SE will bedescribed with reference to FIG. 14. In step SE1, the CPU 34 c checkswhether or not the [template registration] button has been pressed bythe user. If the [template registration] button has been pressed, theprocess proceeds to step SE2. If the [template registration] button hasnot been pressed, the process proceeds to step SF.

In step SE2, the CPU 34 c records the frame image, which is recorded inthe RAM 34 a, as a template image in the RAM 34 a. The template imagerecorded in RAM 34 a is overwritten whenever the [template registration]button is pressed. In step SE3, the CPU 34 c performs a processing fordisplaying the frame image recorded in the RAM 34 a in the [templateimage] box. Specifically, the CPU 34 c performs a processing ofsuperimposing the frame image recorded in the RAM 34 a on a graphicimage signal and outputs a signal after the processing (display signal)to the monitor 22.

It can be seen from the above that the processing in steps SE2 and SE3is processing of registering a frame image, which is captured at thetiming when the [template registration] button is pressed, as a templateimage. In step SE4, the CPU 34 c validates the [record start] button.After the processing in step SE4 ends, the process proceeds to step SF.

Next, the flow of record processing in step SF will be described withreference to FIG. 15. In step SF1, the CPU 34 c checks whether or notthe [record start] button has been pressed by the user. If the [recordstart] button has been pressed, the process proceeds to step SF2. If the[record start] button has not been pressed, the process proceeds to stepSF9.

In step SF2, the CPU 34 c makes the [preview stop] button invalid, the[template registration] button invalid, the [record start] buttoninvalid, the [record stop] button valid, the [image browse] buttoninvalid, and the [maximum record number] box invalid. In step SF3, theCPU 34 c records in the RAM 34 a a record number R, a correlation valueC, a maximum correlation value Cm, a correlation value buffer Cb, and acorrelation value status Sc all of which are set to 0 (R=0, C=0, Cm=0,Cb=0, Sc=0). Details of the correlation value C, the maximum correlationvalue Cm, the correlation value buffer Cb, and the correlation valuestatus Cd will be described later.

In step SF4, the CPU 34 c performs a processing for displaying therecord number R (=0) in the [record number] box. In step SF5, the CPU 34c acquires the maximum record number Rm input in the [maximum recordnumber] box and records it in the RAM 34 a. In step SF6, the CPU 34 ccreates a save folder in the memory card 50. In this case, the date andtime the [record start] button is pressed by the user becomes the foldername of the save folder.

In step SF7, the CPU 34 c saves the template image recorded in the RAM34 a in step SE2 of FIG. 14, as an image file (hereinafter, described asa template image file), in the save folder in the memory card 50. Inthis case, the template image file name becomes “Temp.jpg”. In step SF8,the CPU 34 c sets a record flag to ON and records it in the RAM 34 a.

In step SF9, the CPU 34 c checks whether or not the record flag recordedin the RAM 34 a is ON. If the record flag is ON, the process proceeds tostep SF10. If the record flag is OFF, the process proceeds to step SF18.

In step SF10, the CPU 34 c calculates the correlation value between thetemplate image and the frame image and executes correlation processingfor determining the timing when the record image is saved on the basisof the correlation value. Details of the correlation processing will bedescribed later. In step SF11, the CPU 34 c checks whether or not thesave flag recorded in the RAM 34 a is ON. If the save flag is ON, theprocess proceeds to step SF12. If the save flag is OFF, the processproceeds to step SF18.

In step SF12, the CPU 34 c performs a processing for displaying therecord image, which is recorded in the RAM 34 a during the correlationprocessing in step SF10, in the [record image] box. In step SF13, theCPU 34 c increments the record number R by 1 (R+1 is substituted for R)and records it in the RAM 34 a. In step SF14, the CPU 34 c performs aprocessing for displaying the record number R in the [record image] box.

In step SF15, the CPU 34 c saves the record image, which is recorded inthe RAM 34 a during the correlation processing in step SF10, as an imagefile in the save folder. In step SF16, the CPU 34 c sets a save flag toOFF and records it in the RAM 34 a. In step SF17, the CPU 34 c checkswhether or not the record number R is equal to or larger than themaximum record number Rm. (R≧Rm). If the record number R is equal to orlarger than the maximum record number Rm, the process proceeds to stepSF19. If the record number R is smaller than the maximum record numberRm, the process proceeds to step SF18.

In step SF18, the CPU 34 c checks whether or not the [record stop]button has been pressed by the user. If the [record stop] button hasbeen pressed, the process proceeds to step SF19. If the [record stop]button has not been pressed, the process proceeds to step SG.

In step SF19, the CPU 34 c makes the [preview stop] button valid, the[template registration] button valid, the [record start] button valid,the [record stop] button invalid, the [image browse] button valid, andthe [maximum record number] box valid. In step SF20, the CPU 34 c sets arecord flag to OFF and records it in the RAM 34 a. After the processingin step SF20 ends, the process proceeds to step SG.

Next, the flow of correlation processing in step SF10 will be describedwith reference to FIG. 16. The correspondence relationship between thecorrelation processing shown in FIG. 16 and an actual correlation valuechange will be described later with reference to FIG. 17. In step SF100,the CPU 34 c acquires the luminance value (brightness value) of eachpixel of the template image and the flame image recorded in the RAM 34a. Here, the luminance value of a pixel expressed with the luminance ofeach component of RGB is calculated using the following expression (1),for example.Y=0.299×R+0.587×G+0.114×B  (1)

In step SF101, the CPU 34 c calculates the correlation value C betweenthe frame image and the template image recorded in the RAM 34 a.Hereinafter, details of the correlation value C will be described.Assuming that the luminance values of pixel positions (x, y) of certaintwo images are f1(x, y) and f2(x, y), the average luminance values ofthe two images are expressed as expressions (2) and (3), respectively.In this case, X and Y are the number of pixels in the x and ydirections, respectively, and Size is a total pixel number (Size=X×Y).

$\begin{matrix}{{\overset{\_}{f}}_{1} = \frac{\sum\limits_{y = 1}^{Y}{\sum\limits_{x = 1}^{X}{f_{1}\left( {x,y} \right)}}}{Size}} & (2) \\{{\overset{\_}{f}}_{2} = \frac{\sum\limits_{y = 1}^{Y}{\sum\limits_{x = 1}^{X}{f_{2}\left( {x,y} \right)}}}{Size}} & (3)\end{matrix}$

In addition, standard deviations of the two images are expressed asexpressions (4) and (5), respectively.

$\begin{matrix}{{StdDev}_{1} = \sqrt{\frac{\sum\limits_{y = 1}^{Y}{\sum\limits_{x = 1}^{X}\left( {{f_{1}\left( {x,y} \right)} - {\overset{\_}{f}}_{1}} \right)^{2}}}{Size}}} & (4) \\{{StdDev}_{2} = \sqrt{\frac{\sum\limits_{y = 1}^{Y}{\sum\limits_{x = 1}^{X}\left( {{f_{2}\left( {x,y} \right)} - {\overset{\_}{f}}_{2}} \right)^{2}}}{Size}}} & (5)\end{matrix}$

In addition, the covariance of the two images is expressed as anexpression (6).

$\begin{matrix}{{{Co}{Var}} = \frac{\sum\limits_{y = 1}^{Y}{\sum\limits_{x = 1}^{X}{\left( {{f_{1}\left( {x,y} \right)} - {\overset{\_}{f}}_{1}} \right)\left( {{f_{2}\left( {x,y} \right)} - {\overset{\_}{f}}_{2}} \right)}}}{Size}} & (6)\end{matrix}$

In addition, the correlation value C of the two images is expressed asan expression (7). This correlation value C is an index which indicateswhether or not the two images are similar. Generally, the correlationvalue is a value close to 1 if similar and tends to approach 0 if notsimilar.

$\begin{matrix}{C = \frac{{Co}{Var}}{{StdDev}_{1} \cdot {StdDev}_{2}}} & (7)\end{matrix}$

In the case of calculating the correlation value after thinning out theimage size, it is preferable to change the increased step number of xand y when calculating the total sum regarding x and y and to change thetotal pixel number Size in the above expressions. For example, in thecase of calculating the correlation value after thinning out the imagesize to ¼, it is preferable to set the increased step number of x and yto 4 and to set the total pixel number Size to Size=(X×Y)/(4×4). This iseffective for the case where the speed of correlation processing needsto be improved, since the amount of calculation is reduced if athinning-out processing is used.

In step SF102, the CPU 34 c checks whether or not the correlation valuestatus Sc is 0 (Se=0). The correlation value status Sc is a status ofthe correlation value C. The correlation value status Sc has values of 0to 2. The case where the correlation value status Sc is 0 is an initialstate. The case where the correlation value status Sc is 1 is a stateuntil the CPU 34 c finds a frame image to be saved as a record image.The case where the correlation value status Sc is 2 is a state when theCPU 34 c has found a frame image to be saved as a record image. When thecorrelation value status Sc is 0, the process proceeds to step SF103.When the correlation value status Sc is not 0, the process proceeds tostep SF105.

In step SF103, the CPU 34 c checks whether or not the correlation valueC is larger than the correlation threshold value Ct (C>Ct) and thecorrelation value buffer Cb is equal to or smaller than the correlationthreshold value Ct (Cb≦Ct). The correlation threshold value Ct is athreshold value of the correlation value C, and a predetermined value isrecorded in the RAM 34 a as the correlation threshold value Ct. Thecorrelation value status Sc changes according to which value thecorrelation value C has compared with the correlation threshold valueCt. It will be described later which value is set as the correlationthreshold value Ct. The correlation value buffer Cb is a value in abuffer which is provided in the RAM 34 a in order to hold thecorrelation value C calculated immediately before by the CPU 34 c. IfC>Ct and Cb≦Ct in step SF103, the process proceeds to step SF104. IfC≦Ct or Cb>Ct in step SF103, the process proceeds to step SF105.

In step SF104, the CPU 34 c sets the correlation value status Sc to 1(Sc=1) and records it in the RAM 34 a. In step SF105, the CPU 34 cchecks whether or not the correlation value status Sc is 1 (Sc=1). Whenthe correlation value status Sc is 1, the process proceeds to stepSF106. When the correlation value status Sc is not 1, the processproceeds to step SF110.

In step SF106, the CPU 34 c checks whether or not the correlation valueC is larger than the maximum correlation value Cm (C>Cm). The maximumcorrelation value Cm is a value of a buffer for holding the maximumvalue of the correlation value C. The process proceeds to step SF107 ifC>Cm, and the process proceeds to step SF108 if C≦Cm.

In step SF107, the CPU 34 c sets the maximum correlation value Cm to thecorrelation value C (Cm=C) and records it in the RAM 34 a. In stepSF109, the CPU 34 c records a frame image as a buffer image in the RAM34 a. The buffer image recorded in the RAM 34 a is overwritten wheneverprocessing in step SF109 is executed. The buffer image is an image in abuffer which is provided in the RAM 34 a in order to hold a frame imagetemporarily until the CPU 34 c can check that the frame image is arecord image (image highly correlated with a template image).

In step SF108, the CPU 34 c sets the correlation value status Sc to 2(Sc=2) and records it in the RAM 34 a. In step SF110, the CPU 34 cchecks whether or not the correlation value status Sc is 2 (Sc=2). Whenthe correlation value status Sc is 2, the process proceeds to stepSF111. When the correlation value status Sc is not 2, the processproceeds to step SF116.

In step SF111, the CPU 34 c checks whether or not the correlation valueC is smaller than the correlation threshold value Ct (C<Ct) and thecorrelation value buffer Cb is equal to or larger than the correlationthreshold value Ct (Cb≧Ct). If C<Ct and Cb≧Ct in step SF111, the processproceeds to step SF112. If C≧Ct or Cb<Ct in step SF111, the processproceeds to step SF116.

In step SF112, the CPU 34 c sets the correlation value status Sc to 0(Sc=0) and records it in the RAM 34 a. In step SF113, the CPU 34 c setsthe maximum correlation value Cm to 0 (Cm=0) and records it in the RAM34 a. In step SF114, the CPU 34 c records a buffer image as a recordimage in the RAM 34 a. The record image recorded in the RAM 34 a isoverwritten whenever processing in step SF114 is executed. In stepSF115, the CPU 34 c sets a save flag to ON and records it in the RAM 34a.

In step SF116, the CPU 34 c sets the correlation value buffer Cb to thecorrelation value C (Cb=C) and records it in the RAM 34 a. After theprocessing in step SF116 ends, the process proceeds to step SF11.

FIG. 17 is a graph showing a temporal change of the correlation value C.Hereinafter, details of record processing and correlation processingwill be described with reference to FIG. 17.

The horizontal axis in the graph shown in FIG. 17 indicates a time, andthe vertical axis indicates the correlation value C calculated by theCPU 34 c in step SF101. A maximum and a minimum appear periodically atthe correlation value C. A region where the correlation value C is amaximum indicates a correlation value between a template image and ablade image (image obtained by imaging blades). In addition, a regionwhere the correlation value C is a minimum indicates a correlation valuebetween a template image and the background (inner wall and the like ofa jet engine) of a blade. The correlation threshold value Ct is set tobecome an approximately middle value of both and is recorded in the RAM34 a.

First, the correlation value status Sc is 0 (Sc=0) from timing (t=0), atwhich a user presses the [record start] button, to timing (t=t1), atwhich the correlation value C becomes larger than the correlationthreshold value Ct. Then, the correlation value status Sc is 1 (Sc=1)from t=t1 to timing (t=t2) at which the correlation value C is amaximum. During this period, the maximum correlation value Cm issequentially updated to the correlation value C (Cm=C: step SF107), andframe images are sequentially recorded as buffer images in the RAM 34 a(step SF109).

Then, the correlation value status Sc is 2 (Sc=2) from t=t2 to timing(t=t3) at which the correlation value C becomes smaller than thecorrelation threshold value Ct. During this period, the maximumcorrelation value Cm is not updated and stays fixed, and a frame imageis not recorded as a buffer image in the RAM 34 a.

Then, in t=t3, the correlation value status Sc becomes 0 again (Sc=0)(step SF112), and the buffer image is recorded as a record image in theRAM 34 a (step SF114). In this case, the buffer image is a frame imageat a timing when the correlation value C is the maximum at t=t2. Then,until the user presses the [record stop] button, frame images at timing(t=t4, t5, t6, . . . ) when the correlation value C is a maximum aresequentially saved as record images.

Next, the flow of image browse processing in step SG will be describedwith reference to FIG. 18. In step SG1, the CPU 34 c checks whether ornot the [image browse] button has been pressed by the user. If the[image browse] button has been pressed, the process proceeds to stepSG2. If the [image browse] button has not been pressed, the processproceeds to step SH.

In step SG2, the CPU 34 c performs a processing for displaying an [imagebrowse] window. As described above, a user operation on the main windowis invalid while the [image browse] window is being displayed. In stepSG3, the CPU 34 c performs initialization processing. The initializationprocessing is a processing of setting the initial states of various GUIswithin the [image browse] window or processing of setting the initialvalues of various kinds of data recorded in the RAM 34 a. Details of theinitialization processing will be described later.

In step SG4, the CPU 34 c performs date and time selection processing.The date and time selection processing is a processing in which the CPU34 c detects that the user has changed the selection of a record startdate and time in the [date and time selection] box and changes an imagedisplayed in the [browse image] box. Details of the date and timeselection processing will be described later.

In step SG5, the CPU 34 c performs image selection processing. The imageselection processing is processing in which the CPU 34 c detects thatthe user has pressed the [<<back] button or the [next>>] button andchanges an image displayed in the [browse image] box. Details of theimage selection processing will be described later.

In step SG6, the CPU 34 c checks whether or not the [close] button hasbeen pressed by the user. If the [close] button has been pressed, theprocess proceeds to step SG7. If the [close] button has not beenpressed, the process proceeds to step SG4. In step SG7, the CPU 34 cperforms a processing for making the [image browse] window not bedisplayed. After the processing in step SG7 ends, the process proceedsto step SH.

Next, the flow of initialization processing in step SG3 will bedescribed with reference to FIG. 19. In step SG300, the CPU 34 c createsa save folder list. In step SG301, the CPU 34 c records the created savefolder list in the RAM 34 a. The save folder list recorded in the RAM 34a is overwritten whenever a save folder list is created.

In step SG302, the CPU 34 c creates an image file list in a save folder,of which save folder No is 1, in the save folder list. In step SG303,the CPU 34 c records the created image file list in the RAM 34 a. Theimage folder list recorded in the RAM 34 a is overwritten whenever animage folder list is created.

In step SG304, the CPU 34 c performs a processing for displaying, in the[date and time selection] box, a list of all record start dates andtimes in the save folder list. In step SG305, the CPU 34 c performs aprocessing for highlighting a record start date and time in the savefolder list, which corresponds to save folder No of 1, among the recordstart dates and times displayed in a list in the [date and timeselection] box.

In step SG306, the CPU 34 c performs a processing for displaying animage file, of which image file No is 1 in the image file list, in the[browse image] box. In step SG307, the CPU 34 c performs a processingfor displaying an image file, of which image file No is 1 in the imagefile list, in the [image file name] box.

In step SG308, the CPU 34 c performs a processing for displaying in the[number of image files] box the number of image files in a save folder,of which save folder No is 1 within the save folder list. In step SG309,the CPU 34 c performs a processing for displaying in the [save date andtime] box save date and time of an image file, of which image file No is1 in the image file list. After the processing in step SG309 ends, theprocess proceeds to step SG4.

Next, the flow of date and time selection processing in step SG4 will bedescribed with reference to FIG. 20. In step SG400, the CPU 34 c checkswhether or not the selection of record start date and time in the [dateand time selection] box has been changed by the user. If the selectionof record start date and time has been changed, the process proceeds tostep SG401. If the selection of a record start date and time has notbeen changed, the process proceeds to step SG5.

In step SG401, the CPU 34 c acquires the save folder No of a savefolder, which has a record start date and time that the user hasselected in the [date and time selection] box, from the save folderlist. In this case, the acquired folder number is set to F. In stepSG402, the CPU 34 c creates an image file list in a save folder, ofwhich the save folder No is F, in the save folder list. In step SG403,the CPU 34 c records the image file list created in step SG402 in theRAM 34 a. The image file list recorded in the RAM 34 a is overwrittenwhenever an image file list is created.

In step SG404, the CPU 34 c performs a processing for displaying in the[browse image] box an image file, of which image file No is 1 in theimage file list. In step SG405, the CPU 34 c performs a processing fordisplaying an image file name, of which the image file No is 1 in theimage file list in the [image file name] box.

In step SG406, the CPU 34 c performs a processing for displaying in the[number of image files] box the number of image files in a save folder,of which the save folder No is 1 within the save folder list. In stepSG407, the CPU 34 c performs a processing for displaying in the [savedate and time] box save date and time of an image file, of which imagefile No is 1 in the image file list. After the processing in step SG407ends, the process proceeds to step SG5.

Next, the flow of image selection processing in step SG5 will bedescribed with reference to FIG. 21. In step SG500, the CPU 34 c checkswhether or not the [<<back] button has been pressed by the user. If the[<<back] button has been pressed, the process proceeds to step SG501. Ifthe [<<back] button has not been pressed, the process proceeds to stepSG504.

In step SG501, the CPU 34 c performs a processing for displaying in the[browse image] box an image file, which has image file No that issmaller by 1 than image file No of an image file displayed in thecurrent [browse image] box in the image file list. In step SG502, theCPU 34 c performs a processing for displaying in the [image file name]box an image file name of the image file, which has image file No thatis smaller by 1 than image file No of the currently displayed image filein the image file list.

In step SG503, the CPU 34 c performs a processing for displaying in the[save date and time] box the save date and time of an image file, whichhas image file No smaller by 1 than the image file No of the currentlydisplayed image file in the image file list. In step SG504, the CPU 34 cchecks whether or not the [next>>] button has been pressed by the user.If the [next>>] button has been pressed, the process proceeds to stepSG505. If the [next>>] button has not been pressed, the process proceedsto step SG6.

In step SG505, the CPU 34 c performs a processing for displaying in the[browse image] box an image file, which has image file No that is largerby 1 than the image file No of an image file displayed in the current[browse image] box in the image file list. In step SG506, the CPU 34 cperforms a processing for displaying in the [image file name] box animage file name of the image file, which has an image file No that islarger by 1 than the image file No of the currently displayed image filein the image file list.

In step SG507, the CPU 34 c performs a processing for displaying in the[save date and time] box the save date and time of an image file, whichhas image file No that is larger by 1 than image file No of thecurrently displayed image file in the image file list. After theprocessing in step SG507 ends, the process proceeds to step SG6.

As a modification of the present embodiment, it is also possible toprovide a means for identifying the individual jet engine 1, store inthe endoscope apparatus 3 the maximum number of the turbine blade 10 forevery jet engine, and use the maximum number corresponding to theidentified jet engine 1 at the time of operation of blade recordingsoftware. As a means for identifying the individual jet engine 1, forexample, a bar code or an IC tag may be attached to the jet engine 1.Then, a reader, such as a bar code reader or an IC tag reader, may beconnected to the endoscope apparatus 3 so that the identificationinformation of the jet engine 1 can be read from the bar code or the ICtag using the reader.

According to the present embodiment, the following effects can beacquired. In the present embodiment, a frame image when the position orangle of a turbine blade in a frame image become equal to the positionor angle of a turbine blade in a template image can be acquired byselecting some frame images from a plurality of frame images, which areobtained by imaging turbine blades, on the basis of the correlationvalue which is a result of image comparison between the frame images andthe template image. Images of turbine blades can be acquired by a simplemethod without requiring a special control for matching the rotation ofturbine blades with the imaging timing.

Using the turbine blade image acquired by the method illustrated in thepresent embodiment, it is possible to inspect a turbine blade.Particularly by displaying a record image in the record image box 603shown in FIG. 6, a turbine blade can be inspected in real time. Inaddition, a time for which inspection can be performed or the locationwhere inspection can be performed can be extended by saving the recordimage as an image file in a recording medium. When saving a turbineblade image, the file size becomes large if an endoscope image is savedas a video file as it is. However, if some frame images among endoscopeimages are saved as still image files like the present embodiment aturbine blade images required for inspection can be saved whilepreventing an increase in the required storage capacity of a recordingmedium.

Moreover, by acquiring a frame image when the position or angle of aturbine blade in a frame image becomes equal to the position or angle ofa turbine blade in a template image using the template image as areference image, it is possible to acquire a frame image which is imagedin a state suitable for inspecting a turbine blade by the user. As aresult, the inspection can be performed efficiently. In addition, sincea template image selected from frame images is used, a temporal changein the correlation value shown in FIG. 17 is clear. Accordingly, it ispossible to improve the precision when acquiring a frame image obtainedby imaging in a desired state. In addition, since a template imageselected from frame images is displayed, the user can check whether ornot the state of a turbine blade in the acquired frame image is a statesuitable for the user.

Second Embodiment

Next, a second embodiment of the invention will be described. Althoughonly a browse function of a record image file is set on the [imagebrowse] window of the blade recording software in the first embodiment,not only the browse function of a record image file but also a bladedefect extracting function and a stereo measurement function are set onthe [image browse] window in the present embodiment.

FIG. 22 shows an [image browse] window in the present embodiment. An[image browse] window 2200 shown in FIG. 22 is different from the [imagebrowse] window 700 (FIG. 7) in the first embodiment in that a [defectinspection] group box 2201 is disposed on the right side of the [imagebrowse] window 2200. Various kinds of GUIs for performing defectextraction and stereo measurement are disposed in the [defectinspection] group box 2201. The following explanation will be focused onthe case where the browse image 2202 is a pair of left and right images,which is imaged through a stereo optical adapter capable of forming twosubject images regarding the same subject. The stereo optical adapter ismounted at the tip of the endoscope insertion section 20. Hereinafter,an image displayed on the left side is described as a left image, and animage displayed on the right side is described as a right image.

Hereinafter, functions of various kinds of GUIs in the [defectextraction] group box 2201 will be described. A [defect extraction]check box 2210 is a check box for performing defect extractionprocessing on the browse image 2202. If a user puts a check mark in the[defect extraction] check box 2210, a defect contour 2230 issuperimposed on the browse image 2202 as shown in FIG. 23. Details ofthe defect extraction processing will be described later. Here, anoperation in which the defect contour 2230 is superimposed on the browseimage 2202 as shown in FIG. 23 after the user puts a check mark in the[defect extraction] check box 2210 is performed by a defect extractingsection 34 c ₄ of the CPU 34 c.

A [luminance threshold value] bar 2211 is a bar for setting theluminance threshold value which is one of the inspection parameters inthe defect extraction processing to be described later. The luminancethreshold value is used when binarizing the browse image 2202 in thedefect extraction processing. An [area threshold value] bar 2212 is abar for setting the area threshold value which is one of the inspectionparameters in the defect extraction processing to be described later.The area threshold value is used when removing a small blob (particle)within a browse image in the defect extraction processing. A [luminanceselection] radio button 2213 is a radio button for setting the type ofluminance value which is one of the inspection parameters in the defectextraction processing to be described later. The luminance value is usedwhen converting an image into a gray-scale image in the defectextraction processing.

A [stereo measurement] check box 2220 is a check box for performingstereo measurement, which will be described later, on the browse image2202. If the user puts a check mark in the [stereo measurement] checkbox 2220 in a state where the [defect extraction] check box 2210 ischecked, a measurement region line 2240 is superimposed on the browseimage 2202 as shown in FIG. 24, such that the defect extracted by thedefect extraction processing can be subjected to stereo measurement.

The measurement region line 2240 is a borderline of the region where thestereo measurement can be performed in the browse image 2202, and isdisplayed as a pair of left and right rectangle lines. In addition, if auser moves a cursor 2250 to the defect contour 2230 superimposed on aleft image of the browse image 2202 and designates the defect contour2230 by left clicking or the like as shown in FIG. 25, the defectcontour 2230 is surrounded by a defect rectangle line 2260 and ameasurement point 2270 is displayed on the left image and a matchingpoint 2271 is displayed on the right image as shown in FIG. 26. Detailsof a defect rectangle line, a measurement point, and a matching pointwill be described later. In addition, a result of stereo measurementregarding the designated defect is displayed in a [measurement result]box 2222 to be described later. This operation in which a result ofstereo measurement regarding the designated defect is displayed in the[measurement result] box 2222 after the user put a check mark in the[stereo measurement] check box 2220 is performed by a measurementsection 34 c ₅ of the CPU 34 c. Here, the operation in which the usermoves the cursor 2250 to the defect contour 2230, which is superimposedon the left image of the browse image 2202, and designates the defectcontour 2230 by left clicking or the like, that is, an instruction toselect among the defects extracted by the defect extracting section 34 c₄ is made by inputting an instruction to an input unit 100 such as amouse which is connected to the PC6.

An [environmental data] button 2221 shown in FIG. 22 is a button forselecting the environmental data. The environmental data is data usedwhen performing stereo measurement and includes data for correctingoptical distortion of a stereo optical adapter. The environmental datais the same as that disclosed in Japanese Unexamined Patent Application,First Publication No. 2001-275934.

If the [environmental data] button 2221 is pressed, a file selectiondialog (not shown) is opened. Then, a user selects the environmentaldata on the file selection dialog. The environmental data selected atthis time is data corresponding to a stereo optical adapter used whenimaging an image. Moreover, the [stereo measurement] check box 2220changes from an invalid state to a valid state, so that it becomespossible to put a check mark in the [stereo measurement] check box 2220.

In addition, when the browse image is not an image for stereomeasurement (a pair of left and right images), the [stereo measurement]check box and the [environmental data] button become always invalid sothat the stereo measurement cannot be performed.

The [measurement result] box 2222 is a box for displaying a measurementresult. There are five kinds of measurement results including adistance, widths 1 and 2, a peripheral length, and an area. Details ofthe measurement result will be described later.

Next, the flow of image browse processing in the present embodiment willbe described with reference to FIG. 27. The contents of initializationprocessing in step SG3 a, the date and time selection processing in stepSG4 a, and image selection processing in step SG5 a shown in FIG. 27 aredifferent from the flow (FIG. 18) of the image browse processing in thefirst embodiment. In addition, the flow of image browse processing inthe present embodiment is also different from the flow (FIG. 18) of theimage browse processing in the first embodiment in that defectextraction processing in step SG8, stereo measurement preprocessing instep SG9, and defect designation processing in step SG10 are addedbetween step SG5 a and step SG6 shown in FIG. 27. Hereinafter, only adifferent point from the flow (FIG. 18) of the image browse processingin the first embodiment will be described.

In step SG3 a, the CPU 34 c performs initialization processing. Detailsof the initialization processing will be described later. In step SG4 a,the CPU 34 c performs date and time selection processing. Details of thedate and time selection processing will be described later. In step SG5a, the CPU 34 c performs image selection processing. Details of theimage selection processing will be described later.

In step SG8, the CPU 34 c performs defect extraction processing. Thedefect extraction processing is a processing for extracting a defect ona browse image on the basis of a set inspection parameter andsuperimposing the extracted defect on the browse image. Details of thedefect extraction processing will be described later.

In step SG9, the CPU 34 c performs stereo measurement preprocessing. Thestereo measurement preprocessing is a processing of correcting a browseimage on the basis of selected environmental data so that stereomeasurement of the browse image is possible. Details of the stereomeasurement preprocessing will be described later.

In step SG10, the CPU 34 c performs defect designation processing. Thedefect designation processing is a processing in which the CPU 34 cdetects that a user has designated a defect superimposed on the browseimage and displays a measurement result of a defect size in the[measurement result] box 2222. Details of the defect designationprocessing will be described later.

Next, the flow of initialization processing in step SG3 a will bedescribed with reference to FIG. 28. The point that steps SG310 andSG311 are added after step SG309 shown in FIG. 28 is different from theflow (FIG. 19) of the initialization processing in the first embodiment.Hereinafter, only a different point from the flow (FIG. 19) of theinitialization processing in the first embodiment will be described.

In step SG310, the CPU 34 c invalidates a [stereo measurement] checkbox. In step SG311, the CPU 34 c sets a defect extraction flag and astereo measurement flag to OFF and records them in the RAM 34 a. Thedefect extraction flag is a flag indicating whether or not to performthe defect extraction processing. The stereo measurement flag is a flagindicating whether or not to perform the stereo measurementpreprocessing. Next, the flow of date and time selection processing instep SG4 a will be described with reference to FIG. 29. The point thatstep SG408 is added after step SG407 shown in FIG. 29 is different fromthe flow (FIG. 20) of the date and time selection processing in thefirst embodiment. Hereinafter, only a different point from the flow(FIG. 20) of the date and time selection processing in the firstembodiment will be described.

In step SG408, the CPU 34 c sets a defect extraction flag and a stereomeasurement flag to ON and records them in the RAM 34 a. The reason whythe defect extraction flag and the stereo measurement flag are set to ONin step SG408 is that if the selection of record start date and time ofa [date and time selection] box is changed in step SG400, it isnecessary to perform the defect extraction processing and the stereomeasurement preprocessing again since the browse image is changed.Moreover, when an image file is displayed in the [browse image] box instep SG404, all measurement region lines and the like which are alreadysuperimposed are not displayed.

Next, the flow of image selection processing in step SG5 will bedescribed with reference to FIG. 30. The point that step SG508 is addedafter step SG503 shown in FIG. 30 and step SG509 is added after stepSG507 is different from the flow (FIG. 21) of the image selectionprocessing in the first embodiment. Hereinafter, only a different pointfrom the flow (FIG. 21) of the image selection processing in the firstembodiment will be described.

In step SG508, the CPU 34 c sets a defect extraction flag and a stereomeasurement flag to ON and records them in the RAM 34 e. In step SG509,the CPU 34 c sets a defect extraction flag and a stereo measurement flagto ON and records them in the RAM 34 a. The reason why the defectextraction flag and the stereo measurement flag are set to ON in stepsSG508 and SG509 is that if the [<<back] button and the [next>>] buttonare pressed in steps SG500 and SG504, it is necessary to perform thedefect extraction processing and the stereo measurement preprocessingagain since the browse image is changed. Moreover, when an image file isdisplayed in the [browse image] box in steps SG501 and SG505, allmeasurement region lines and the like which are already superimposed arenot displayed.

Next, the flow of defect extraction processing in step SG8 will bedescribed with reference to FIGS. 31 and 32. In step SG800, the CPU 34 cchecks whether or not the [defect extraction] check box has beenchecked. If the [defect extraction] check box has been checked, theprocess proceeds to step SG801. If the [defect extraction] check box isnot checked, the process proceeds to step SG802.

In step SG801, the CPU 34 c acquires the luminance threshold value Yt,the area threshold value At, and the luminance selection S from a[luminance threshold value] bar, an [area threshold value] bar, and a[luminance selection] radio button, respectively, and records them inthe RAM 34 a. In step SG802, the CPU 34 c checks whether or not therehas been an instruction from the user to put a check mark in the [defectextraction] check box. If there has been an instruction to put a checkmark in the [defect extraction] check box, the process proceeds to stepSG803. If there is no instruction to put a check mark in the [defectextraction] check box, the process proceeds to step SG9.

In step SG803, similar to step SG801, the CPU 34 c acquires theluminance threshold value Yt, the area threshold value At, and theluminance selection S from a [luminance threshold value] bar, an [areathreshold value] bar, and a [luminance selection] radio button,respectively, and records them in the RAM 34 a. In addition, the CPU 34c performs a processing of putting a check mark in the [defectextraction] check box.

In step SG804, the CPU 34 c sets the luminance threshold value Yt, thearea threshold value At, and the luminance selection S, which wereacquired in step SG803, as a previous luminance threshold value Yt1, aprevious area threshold value At1, and a previous luminance selectionS1, respectively, and records them in the RAM 34 a. The luminancethreshold value Yt, the area threshold value At, and the luminanceselection S used when performing the previous defect extractionprocessing are temporarily recorded as the previous luminance thresholdvalue Yt1, the previous area threshold value At1, and the previousluminance selection S1, respectively. In step SG805, the CPU 34 c sets adefect extraction flag to ON and records it in the RAM 34 a.

In step SG806, the CPU 34 c checks whether or not the luminancethreshold value Yt is equal to the previous luminance threshold valueYt1, whether or not the area threshold value At is equal to the previousarea threshold value At1, and whether or not the luminance selection Sis equal to the previous luminance selection S1. The processing in stepSG806 is processing of checking whether or not inspection parametersused when performing the previous defect extraction processing have beenchanged by the user. If all inspection parameters are equal to thoseused when performing the previous defect extraction processing, theprocess proceeds to step SG808. If one or more inspection parameters aredifferent from those used when performing the previous defect extractionprocessing, the process proceeds to step SG807. In step SG807, the CPU34 c sets a defect extraction flag to ON and records it in the RAM 34 a.

In step SG808, the CPU 34 c checks whether or not the defect extractionflag is ON. If the defect extraction flag is ON, the process proceeds tostep SG809. If the defect extraction flag is OFF, the process proceedsto step SG821.

Hereinafter, FIG. 33 will also be used appropriately to describe stepsSG809 to SG818. In step SG809, the CPU 34 c acquires the image data ofthe template image file and the browse image file saved in a save folderand records them in the RAM 34 a. The image data refers to the RGBluminance value of each pixel of an image.

In step SG810, the CPU 34 c converts the acquired two image data intogray-scale images on the basis of the luminance selection S recorded inthe RAM 34 a in step SG801 or SG803. When the luminance selection S is“Gray”, the luminance value Y of each pixel of the gray-scale image iscalculated from the RGB luminance value of each pixel of image datausing the following expression (8),Y=0.299×R+0.587×G+0,114×B  (8)

In addition, when the luminance selection S is one of “R”, “G”, and “B”,the luminance value of each of R, G, and B of each pixel of the imagedata becomes a luminance value Y of each pixel of a gray-scale image asit is.

In step SG811, the CPU 34 c creates an image (hereinafter, described asa differential image) corresponding to the difference between the twogray-scale images created in step SG810. (a), (b), and (c) of FIG. 33show a situation where a differential image 3310 is created by taking adifference between a gray-scale image 3300 of the template image and agray-scale image 3301 of the browse image. These series of operationsare performed by a first difference extracting section 34 c ₆ of the CPU34 c.

In step SG812, the CPU 34 c creates a binary image by binarizing thedifferential image on the basis of the luminance threshold value Ytrecorded in the RAM 34 a. (c) and (d) of FIG. 33 show a situation wherea binary image 3320 is created by binarizing the differential image3310.

In step SG813, the CPU 34 c removes a small noise by performingexpansion and contraction processing on the created binary image. Instep SG814, the CTU 34 c extracts a blob (particle) by performinglabeling processing on the binary image from which noise was removed instep SG813. In step SG815, the CPU 34 c removes a blob with an area,which is smaller than the area threshold value At recorded in the RAM 34a, from the image from which noise was removed in step SG814. (d) and(e) of FIG. 33 show a situation where a small blob is removed from thebinary image 3320.

In step SG816, the CPU 34 c extracts the contour of a remaining blob, asa defect contour, from the binary image from which a small blob wasremoved in step SG814. (e) and (d) of FIG. 33 show a situation where acontour 3330 of a blob was extracted. In step SG817, the CPU 34 crecords the coordinates of the defect contour extracted in step SG816 inthe RAM 34 a.

In step SG818, the CPU 34 c performs a processing of superimposing thedefect contour on the browse image on the basis of the coordinates ofthe defect contour recorded in the RAM 34 a. (f) and (g) of FIG. 33 showa situation where the defect contour 3330 is superimposed on a browseimage 3340. In step SG819, the CPU 34 c sets the luminance thresholdvalue Yt, the area threshold value At, and the luminance selection S,which were recorded in the RAM 34 a in step SG801 or SG803, as theprevious luminance threshold value Yt1, the previous area thresholdvalue At1, and the previous luminance selection S1, respectively, andrecords them in the RAM 34 a.

In step SG820, the CPU 34 c sets a defect extraction flag to OFF andrecords it in the RAM 34 a. In step SG821, the CPU 34 c checks whetheror not there has been an instruction from the user to remove a checkmark from the [defect extraction] check box. If there has been aninstruction to remove a check mark from the [defect extraction] checkbox, the process proceeds to step SG822. If there is no instruction toremove a check mark from the [defect extraction] check box, the processproceeds to step SG9.

In step SG822, the CPU 34 c performs a processing of making the defectcontour, which is displayed on the browse image, not displayed on thebasis of the coordinates of the defect contour recorded in the RAM 34 ain step SG817. In addition, the CPU 34 c performs a processing ofremoving a check mark from the [defect extraction] check box. After theprocessing in step SG822 ends, the process proceeds to step SG9.

Next, the flow of stereo measurement preprocessing in step SG9 will bedescribed with reference to FIG. 34. In step SG900, the CPU 34 c checkswhether or not an [environmental data] button has been pressed by theuser. If the [environmental data] button has been pressed, the processproceeds to step SG901. If the [environmental data] button is notpressed, the process proceeds to step SG905.

In step SG901, the CPU 34 c performs a processing for displaying an[open a file] dialog (not shown). In step SG902, the CPU 34 c checkswhether or not the user has selected the environmental data on the [opena file] dialog. If the environmental data has been selected, the processproceeds to step SG903. If the environmental data has not been selected,the process proceeds to step SG905.

In step SG903, the CPU 34 c records the selected environmental data inthe RAM 34 a. The environmental data recorded in the RAM 34 a isoverwritten whenever the environmental data is selected. In step SG904,the CPU 34 c validates the [stereo measurement] check box. In stepSG905, the CPU 34 c checks whether or not there is a check mark in the[stereo measurement] check box. If there is a check mark in the [stereomeasurement] check box, the process proceeds to step SG908. If there isno check mark in the [stereo measurement] check box, the processproceeds to step SG906.

In step SG906, the CPU 34 c checks whether or not there has been aninstruction from the user to put a check mark in the [stereomeasurement] check box. If there has been an instruction to put a checkmark in the [stereo measurement] check box, the process proceeds to stepSG907. If there is no instruction to put a check mark in the [stereomeasurement] check box, the stereo measurement preprocessing (SG9) endsand the process proceeds to step SG10.

In step SG907, the CPU 34 c sets a stereo measurement flag to ON andrecords it in the RAM 34 a. In addition, the CPU 34 c performs aprocessing of putting a check mark in the [stereo measurement] checkbox. In step SG908, the CPU 34 c checks whether or not the stereomeasurement flag is ON. If the stereo measurement flag is ON, theprocess proceeds to step SG909. If the stereo measurement Dag is OFF,the process proceeds to step SG10.

In step SG909, the CPU 34 c performs a processing of superimposing ameasurement region line on the browse image on the basis of thecoordinates of the measurement region line recorded in the RAM 34 a. Thecoordinates of the measurement region line are recorded in the RAM 34 aas a part of the environmental data.

In step SG910, the CPU 34 c acquires the image data of the browse imagefile saved in the save folder and records it in the RAM 34 a. In stepSG911, the CPU 34 c corrects the image data acquired in step SG910. Thecorrection processing performed in step SG911 is the same as thatdisclosed in Japanese Unexamined Patent Application No. H10-248806.

In step SG912, the CPU 34 c records the image data corrected in stepSG911, as correction image data, in the RAM 34 a. The correction imagedata recorded in the RAM 34 a is overwritten whenever the correctionimage data is created. In step SG913, the CPU 34 c sets a stereomeasurement flag to OFF and records it in the RAM 34 a.

In step SG914, the CPU 34 c checks whether or not there has been aninstruction from the user to remove a check mark from the [stereomeasurement] check box. If there has been an instruction to remove acheck mark from the [stereo measurement] check box, the process proceedsto step SG915. If there is no instruction to remove a check mark fromthe [stereo measurement] check box, the process proceeds to step SG10.In step SG915, the CPU 34 c performs a processing of making themeasurement region line, which is displayed on the browse image, notdisplayed on the basis of the coordinates of the measurement region linerecorded in the RAM 34 a. In addition, the CPU 34 c performs aprocessing of removing a check mark from the [stereo measurement] checkbox. After the processing in step SG915 ends, the process proceeds tostep SG10.

Next, the flow of defect designation processing in step SG10 will bedescribed with reference to FIG. 35. In step SG1000, the CPU 34 c checkswhether or not there is a check mark in the [stereo measurement] checkbox. If there is a check mark in the [stereo measurement] check box, theprocess proceeds to step SG1001. If there is no check mark in the[stereo measurement] check box, the process proceeds to step SG6. Instep SG1001, the CPU 34 c checks whether or not the defect contourdisplayed in the left measurement region of the browse image has beendesignated by the user. If the defect contour has been designated by theuser, the process proceeds to step SG1002. If the defect contour is notdesignated by the user, the process proceeds to step SG6.

In step SG1002, the CPU 34 c performs a processing of making the defectrectangle line, the measurement point, and the matching point, which arealready superimposed on the browse image, not be displayed. In stepSG1003, the CPU 34 c performs a processing of superimposing the defectrectangle line on the browse image. The defect rectangle line is arectangle line displayed around a defect region line designated by theuser, and indicates that it is the defect contour currently designatedby the user.

Hereinafter, FIG. 36 will also be used appropriately to describe stepsSG1004 to SG1009. In step SG1004, the CPU 34 c calculates themeasurement point coordinates on the basis of the coordinates of thedefect contour currently designated by the user, which are recorded inthe RAM 34 a. The measurement point is a point used when measuring thesize of a defect. As shown in FIGS. 36A and 36B, measurement points 3610are located with equal distances on a defect contour 3600.

In step SG1005, the CPU 34 c calculates the matching point coordinatesin the right measurement region, which correspond to the measurementpoint coordinates in the left measurement region, on the basis of theimage data of the browse image. More specifically, the CPU 34 ccalculates the coordinates of a matching point, which is correspondingpoints of the two left and right images, by executing pattern matchingprocessing on the basis of the measurement point coordinates. Thispattern matching processing method is the same as that disclosed inJapanese Unexamined Patent Application No. 2004-49638.

In step SG1006, the CPU 34 c calculates the space point coordinates(three-dimensional coordinates in the actual space) of each measurementpoint on the basis of the measurement point coordinates and the matchingpoint coordinates calculated in step SG1004 and SG1005. The method ofcalculating the space point coordinates is the same as that disclosed inJapanese Unexamined Patent Application No. 2004-49638.

In step SG1007, the CPU 34 c calculates a measurement result on thebasis of the space point coordinates calculated in step SG1006. Thereare five kinds of measurement result including a distance, widths 1 and2, a peripheral length, and an area of a defect.

The distance is an average value of coordinates of all space points inthe depth direction. As shown in FIGS. 36C and 36D, the width 1 is aspatial distance between measurement points located nearest tointersections between an equivalent ellipse 3620, which is calculatedfrom all measurement point coordinates, and a long axis 3621 of theellipse 3620. As shown in FIGS. 36C and 36D, the width 2 is a spatialdistance between measurement points located nearest to intersectionsbetween the equivalent ellipse 3620 and a short axis 3622 of the ellipse3620. In addition, the equivalent ellipse is an ellipse which can beapproximated from a plurality of coordinates. The peripheral length isthe sum of space point distances 3630 of all adjacent measurementpoints, as shown in FIG. 36E. The area is a space area of a region 3640surrounded by all adjacent measurement points, as shown in FIG. 36F.

In step SG1008, the CPU 34 c performs a processing of superimposing themeasurement point in the left measurement region of the browse imagewhile superimposing the matching point in the right measurement region.In step SG1009, the CPU 34 c performs a processing of displaying themeasurement result calculated in step SG1007 in the [measurement result]box. After the processing in step SG1009 ends, the process proceeds tostep SG6.

In the present embodiment, a browse image obtained by imaging using thestereo optical adapter is used. However, a browse image obtained byimaging using optical adapters other than the stereo optical adapter mayalso be used in the defect extraction processing. On an [image browse]window 3700 shown in FIG. 37, a browse image 3701 obtained by imagingone subject image, which is formed by the optical adapter, is displayed.If a user puts a check mark in the [defect extraction] check box 3710, adefect contour 3720 is superimposed on the browse image 3701 as shown inFIG. 38.

Moreover, in the present embodiment, the defect contour 2230 isdisplayed at the position, which corresponds to the defect extracted bythe defect extraction processing, on the browse image 2202 as shown inFIG. 23. However, things other than a line may also be displayed as longas they can specify the position of a defect. For example, a figure,such as an arrow, may be displayed at the position corresponding to adefect or a phrase, such as “defect”, may be displayed.

According to the present embodiment, a defect in a blade can beextracted regardless of the kind of defect by extracting the differencebetween a browse image and a template image. In addition, it becomeseasy for a user to recognize the position of a defect by superimposingthe defect contour or the like on the extracted defect.

In addition, the size of a defect can be checked by measuring theextracted defect. In addition, the size of a defect that the user wantsto know can be checked by measuring a defect, which is designated whenthe user designates the defect contour, among defects displayed on thebrowse image. In addition, the three-dimensional size of a defect can bechecked by performing defect extraction processing using a browse image,which is obtained by imaging using a stereo optical adapter, andexecuting stereo measurement on the basis of the extracted defect.

Third Embodiment

Next, a third embodiment of the invention will be described. On an[image browse] window of blade recording software in the presentembodiment, a blade region extracting function is set in addition to thedefect extracting function and the stereo measurement function in thesecond embodiment.

Hereinafter, the flow of operation of the blade recording software inthe present embodiment will be described with reference to FIG. 39. Thecontents of initialization processing in step SC, template registrationprocessing in step SE, and record processing in step SF shown in FIG. 39are different from the flow (FIG. 11) of operation of the bladerecording software in the first embodiment. In addition, the point thatreference image registration processing in step SJ is added betweensteps SD and SE shown in FIG. 39 is also different from the flow (FIG.11) of operation of the blade recording software in the firstembodiment. Hereinafter, only a different point from the flow (FIG. 11)of operation of the blade recording software in the first embodimentwill be described.

In step SC, the CPU 34 c performs initialization processing. Details ofthe initialization processing will be described later.

In step SJ, the CPU 34 c performs reference image registrationprocessing. The reference image is a frame image after one frame from atemplate image. The reference image is used when extracting a bladeregion, and will be described later. Details of reference imageregistration processing will be described later. This reference imageregistration processing is performed by a reference image extractingsection 34 c ₇ of the CPU 34 c.

In step SE, the CPU 34 c performs template registration processing.Details of the template registration processing will be described later.

In step SF, the CPU 34 c performs record processing. Details of therecord processing will be described later.

Next, the flow of initialization processing in step SC will be describedwith reference to FIG. 40. The contents of step SC5 a shown in FIG. 40are different from the flow (FIG. 12) of operation of the bladerecording software in the first embodiment. Hereinafter, the contents ofstep SC5 _(a) will be described.

In step SC5 a, the CPU 34 c sets all of a preview flag, a record flag, asave flag, and a reference image flag to OFF and records them in the RAM34 a. The reference image flag is a flag indicating whether or not areference image is saved as an image file. Similar to the template imagefile, the reference image file is saved in a save folder in the memorycard 50. A file name of the reference image becomes “Ref.jpg”. After theprocessing in step SC5 a ends, the process proceeds to step SD.

Next, the flow of reference image registration processing in step SJwill be described with reference to FIG. 41. In step SJ1, the CPU 34 cchecks whether or not the reference image flag is ON. If the referenceimage flag is ON, the process proceeds to step SJ2. If the referenceimage flag is OFF, the process proceeds to step SE.

In step SJ2, the CPU 34 c records the frame image, which is recorded inthe RAM 34 a, as a reference image in the RAM 34 a. In step SJ3, the CPU34 c validates the [record start] button. In step SJ4, the CPU 34 c setsa reference image flag to OFF and records it in the RAM 34 a. After theprocessing in step SJ4 ends, the process proceeds to step SE.

Next, the flow of template registration processing in step SE will bedescribed with reference to FIG. 42. The point that step SE4 shown inFIG. 14 is changed to step SE5 is different from the flow (FIG. 14) ofoperation of the blade recording software in the first embodiment.Hereinafter, the contents of step SE5 will be described.

In step SE5, the CPU 34 c sets a reference image flag to ON and recordsit in the RAM 34 a. After the processing in step SE5 ends, the processproceeds to step SF. After a frame image of a certain frame is recordedas a template image in the RAM 34 a, the reference image flag is set toON in step SE5. Accordingly, in step SJ2 shown in FIG. 41, a frame imageof the next frame is recorded as a reference image in the RAM 34 a.

Next, the flow of record processing in step SF will be described withreference to FIG. 43. The contents of step SF7 a shown in FIG. 43 aredifferent from the contents of step SF7 of the flow (FIG. 15) ofoperation of the blade recording software in the first embodiment.Hereinafter, the contents of step SF7 a will be described.

In step SF7 a, the CPU 34 c saves the template image and the referenceimage, which are recorded in the RAM 34 a, as image files in a savefolder in the memory card 50.

FIG. 44 shows an [image browse] window in the present embodiment. Apoint, which is different from the [image browse] window (FIG. 22) inthe second embodiment, is that a [blade region extraction] check box4410 is disposed in a [defect inspection] group box 4400. The followingexplanation will be focused on the case where a browse image 4402(display section) is not an image for stereo measurement (a pair of leftand right images).

The [blade region extraction] check box 4410 is a check box forperforming blade region extraction processing on the browse image 4402.If a user puts a check mark in the [blade region extraction] check box4410, a blade region 4420 is superimposed on the browse image 4402 asshown in FIG. 45. In this case, a graphic image showing the blade region4420 is superimposed on the browse image 4402 so that a user can easilydistinguish the blade region 4420 from other regions.

In addition, if a check mark is put in a [defect extraction] check box4430 in a state where a check mark is put in the [blade regionextraction] check box 4410, a defect contour 4440 is superimposed on thebrowse image 4402 only for a defect, which is located in the bladeregion 4420, among the defects extracted by the defect extractionprocessing and the other defect contour is not displayed, as shown inFIG. 46. Details of the blade region extraction processing will bedescribed later.

Next, the flow of image browse processing in the present embodiment willbe described with reference to FIG. 47. The contents of initializationprocessing in step SG3 b, date and time selection processing in step SG4b, image selection processing in step SG5 b, and defect extractionprocessing in step SG8 b are different from the contents of steps SG3 a,SG4 a, SG5 a, and SG8 of the flow (FIG. 27) of the image browseprocessing in the second embodiment. In addition, the point that bladeregion extraction processing in step SG11 is added between step SG5 band step SG8 b shown in FIG. 47 is also different from the flow (FIG.27) of the image browse processing in the second embodiment.Hereinafter, only a different point from the flow (FIG. 27) of the imagebrowse processing in the second embodiment will be described.

In step SG3 b, the CPU 34 c performs initialization processing. Detailsof the initialization processing will be described later. In step SG4 b,the CPU 34 c performs date and time selection processing. Details of thedate and time selection processing will be described later. In step SG5b, the CPU 34 c performs image selection processing. Details of theimage selection processing will be described later.

In step SG11, the CPU 34 c performs blade region extraction processing.The blade region extraction processing is a processing of extracting ablade region by performing the same processing as the defect extractionprocessing on a browse image and superimposing the extracted bladeregion on the browse image. In addition, the blade region extractionprocessing also includes processing of making only a defect, which islocated in a blade region among the defects superimposed on the browseimage, displayed and the other defects not be displayed. Details of theblade region extraction processing will be described later.

In step SG8 b, the CPU 34 c performs defect extraction processing.Details of the defect extraction processing will be described later.

Next, the flow of initialization processing in step SG3 b will bedescribed with reference to FIG. 48. The contents of step SG311 b shownin FIG. 48 are different from the contents of steps SG311 of the flow(FIG. 28) of operation of the blade recording software in the secondembodiment. Hereinafter, the contents of step SG311 b will be described.

In step SG311 b, the CPU 34 c sets a blade region extraction flag, adefect extraction flag, and a stereo measurement flag to OFF and recordsthem in the RAM 34 a. The blade region extraction flag is a flagindicating whether to perform blade region extraction. After theprocessing in step SG311 b ends, the process proceeds to step SG4 b.

Next, the flow of date and time selection processing in step SG4 b willbe described with reference to FIG. 49. The contents of step SG408 bshown in FIG. 49 are different from the contents of SG408 of the flow(FIG. 29) of operation of the blade recording software in the secondembodiment. Hereinafter, the contents of step SG408 b will be described.

In step SG408 b, the CPU 34 c sets a blade region extraction flag, adefect extraction flag, and a stereo measurement flag to ON and recordsthem in the RAM 34 a. After the processing in step SG408 b ends, theprocess proceeds to step SG5 b. The reason why the blade regionextraction flag, the defect extraction flag, and the stereo measurementflag are set to ON in step SG408 b is that if the selection of recordstart date and time of a [date and time selection] box is changed instep SG400, it is necessary to perform the blade region extractionprocessing, the defect extraction processing, and the stereo measurementprocessing again since the browse image is changed.

Next, the flow of image selection processing in step SG5 b will bedescribed with reference to FIG. 50. The contents of steps SG508 b andSG509 b shown in FIG. 50 are different from the contents of steps SG508and SG509 of the flow (FIG. 30) of operation of the blade recordingsoftware in the second embodiment. Hereinafter, the contents of stepsSG508 b and SG509 b will be described.

In step SG508 b, the CPU 34 c sets a blade region, extraction flag, adefect extraction flag, and a stereo measurement flag to ON and recordsthem in the RAM 34 a. In step SG509 b, the CPU 34 c sets the bladeregion extraction flag, the defect extraction flag, and the stereomeasurement flag to ON and records them in the RAM 34 a. After theprocessing in step SG509 b ends, the process proceeds to step SG11.

The reason why the blade region extraction flag, the defect extractionflag, and the stereo measurement flag are set to ON in steps SG508 b andSG509 b is that if the [<<back] button and the [next>>] button arepressed in steps SG500 and SG504, it is necessary to perform the bladeregion extraction processing, the defect extraction processing, and thestereo measurement processing again since the browse image is changed.

Next, the flow of blade region extraction processing in step SG11 willbe described with reference to FIGS. 51 and 52. In step SG1100, the CPU34 c checks whether or not there is a check mark in the [blade regionextraction] check box. If there is a check mark in the [blade regionextraction] check box, the process proceeds to step SG1103. If there isno check mark in the [blade region extraction] check box, the processproceeds to step SG1101.

In step SG1101, the CPU 34 c checks whether or not there has been aninstruction from the user to put a check mark in the [blade regionextraction] check box. If there has been an instruction to put a checkmark in the [blade region extraction] check box, the process proceeds tostep SG1102. If there is no instruction to put a check mark in the[blade region extraction] check box, the process proceeds to step SG8 b.

In step SG1102, the CPU 34 c sets the blade region extraction flag to ONand records it in the RAM 34 a. In addition, the CPU 34 c performs aprocessing of putting a check mark in the [blade region extraction]check box.

In step SG1103, the CPU 34 c checks whether or not the blade regionextraction flag is ON. If the blade region extraction flag is ON, theprocess proceeds to step SG1104. If the blade region extraction flag isOFF, the process proceeds to step SG8 b.

Hereinafter, FIGS. 53 and 54 will also be used appropriately to describesteps SG1104 to SG1116. In step SG1104, the CPU 34 c acquires the imagedata of the template image file and the reference image file saved in asave folder and records them in the RAM 34 a. The image data refers tothe RGB luminance value of each pixel of an image.

In step SG1105, the CPU 34 c converts into gray-scale images the imagedata of two sheets acquired in step SG1104. The luminance value Y ofeach pixel of the gray-scale image is calculated from the RGB luminancevalue of each pixel of image data using the following expression (9).Y0.299×R+0.587×G+0.114×B  (9)

In step SG1106, the CPU 34 c creates an image (hereinafter, described asa differential image) corresponding to the difference between the twogray-scale images created in step SG1105. FIG. 53 shows a situationwhere a differential image 5310 is created by taking a differencebetween a gray-scale image 5300 of a template image and a gray-scaleimage 5301 of a reference image. Since the template image and thereference image deviate from each other by one frame, a difference isextracted in the boundary of a blade region as shown in FIG. 53. Theoperation of taking the difference between the gray-scale image 5300 ofthe template image and the gray-scale image 5301 of the reference image,that is, an operation of taking a difference between a template imageand a reference image is performed by a second difference extractingsection 34 c _(g) of the CPU 34 c.

In step SG1107, the CPU 34 c creates a binary image by binarizing thedifferential image on the basis of a predetermined threshold value. FIG.53 shows a situation where a binary image 5320 is created by binarizingthe differential image 5310.

In step SG1108, the CPU 34 c removes a small noise by performingexpansion and contraction processing on the created binary image. Instep SG1109, the CPU 34 c extracts a blob (particle) by performinglabeling processing on the binary image from which noise was removed instep SG1108. In step SG1110, the CPU 34 c removes a blob with an area,which is smaller than a predetermined area, from the image from whichnoise was removed in step SG1108. FIG. 53 shows a situation where asmall blob is removed from the binary image 5320.

In step SG1111, the CPU 34 c extracts a straight line by performingHough transform on the binary image from which a small blob was removedin step SG1110. In this case, the extracted straight line is assumed tobe a blade borderline. FIG. 54 shows a situation where a bladeborderline 5400 is extracted.

In step SG1112, the CPU 34 c extracts a plurality of regions divided bya blade borderline. FIG. 54 shows a situation where a plurality ofregions A to I divided by the blade borderline 5400 are extracted. Theregions A to I are located in a line in order of regions adjacent toeach other.

In step SG1113, the CPU 34 c calculates the average luminance of eachregion, which was extracted in step SG1112, on the template image. FIG.55 shows a graph of the average luminance of regions A to I. From thisgraph, it can be seen that a region with a high average luminance and aregion with a low average luminance appear alternately. This is becausea blade region with high luminance and a background region with lowluminance are alternately located in a line like the browse image 5410shown in FIG. 54. Only a blade region can be extracted using thisrelationship.

In step SG1114, the CPU 34 c extracts a blade region on the basis of theaverage luminance of each region calculated in step SG1113. For example,the CPU 34 c compares the luminance of two regions adjacent to eachother and sets a region with higher luminance as a blade region and aregion with lower luminance as a background region. The CPU 34 cdetermines whether a corresponding region is a blade region or abackground region while shifting two adjacent regions. Thus, theoperation of extracting (detecting) a blade region on the basis of thedifference between the template image and the reference image isperformed by a detecting section 34 c ₉ of the CPU 34 c.

In step SG1115, the CPU 34 c records the coordinates of the blade regionextracted in step SG1114 in the RAM 34 a. The coordinates of the bladeregion are coordinates of a representative point among the points whichform the blade region, for example. In this case, the coordinates of theblade region recorded in the RAM 34 a are overwritten whenever a bladeregion is extracted. FIG. 54 shows a situation where a blade region 5430is extracted from a browse image 5420.

In step SG1116, the CPU 34 c performs a processing of superimposing theblade region on the browse image on the basis of the coordinates of theblade region recorded in the RAM 34 a. FIG. 54 shows a situation where ablade region 5450 is superimposed on a browse image 5440. The bladeregion may be superimposed as shown in FIG. 54, or a line showing ablade region may be superimposed.

In step SG1117, the CPU 34 c sets the blade region extraction flag toOFF and records it in the RAM 34 a. In step SG1118, the CPU 34 c checkswhether or not there has been an instruction from the user to remove acheck mark from the [blade region extraction] check box. If there hasbeen an instruction to remove a check mark from the [blade regionextraction] check box, the process proceeds to step SG1119. If there isno instruction to remove a check mark from the [blade region extraction]check box, the process proceeds to step SG8 b.

In step SG1119, the CPU 34 c performs a processing of making the bladeregion, which is displayed on the browse image, not displayed on thebasis of the coordinates of the blade region recorded in the RAM 34 a instep SG1115. In addition, the CPU 34 c performs a processing of removinga check mark from the [blade region extraction] check box. After theprocessing in step SG1119 ends, the process proceeds to step SG8 b.

Next, the flow of defect extraction processing in step SG8 b will bedescribed with reference to FIGS. 56 and 51. The point that steps SG823and SG824 are added between steps SG815 and SG816 shown in FIG. 57 isdifferent from the flow (FIGS. 31 and 32) of the defect extractionprocessing in the second embodiment. Hereinafter, the contents of stepsSG823 and SG824 will be described.

In step SG823, the CPU 34 c checks whether or not there is a check markin the [blade region extraction] check box. If there is a check mark inthe [blade region extraction] check box, the process proceeds to stepSG824. If there is no check mark in the [blade region extraction] checkbox, the process proceeds to step SG816.

In step SG824, the CPU 34 c removes a blob located outside the bladeregion on the basis of the coordinates of the blade region recorded inthe RAM 34 a in step SG1115. Accordingly, in step SG816 performedsubsequent to step SG824, the defect contour within the blade region isextracted. This operation, that is an operation for determining whetheror not the defect is within the blade region of the blade image, isperformed by a defect position determining section 34 c ₁₀ of the CPU 34c.

In the present embodiment, a browse image obtained by imaging using anoptical adapter other than a stereo optical adapter is used. However, abrowse image obtained by imaging using the stereo optical adapter mayalso be used. By using the browse image obtained by imaging using thestereo optical adapter, a defect of the blade region can be measured bydefect designation processing in step SG-10.

Moreover, in the present embodiment, a blade region extracted by bladeregion extraction processing is displayed on the browse image 4402 asshown in FIG. 45. However, the display mode of a blade region is notlimited if the position of the blade region can be specified. Forexample, a figure, such as an arrow, may be displayed at the positioncorresponding to the blade region or a phrase, such as “blade”, may bedisplayed.

According to the present embodiment, when a user browses a record imagefile, a blade region can be extracted and only a defect in the bladeregion can be extracted. According to the invention, an image of bladescan be acquired by a simple method without requiring a special controlfor matching the rotation of blades with the imaging timing by selectingsome images from the images, which are obtained by imaging the blades,on the basis of a result of image comparison with the template image. Adefect of a blade can be detected regardless of the kind of defect byextracting the difference between the template image and the imageacquired as described above. In addition, according to the invention, ablade region within a blade image can be detected.

In addition, in accordance with the above-described embodiments, it ispossible to have the following image processing method. The imageprocessing method includes; a step of extracting a template image fromblade images obtained by capturing blades periodically arrayed in a jetengine; a step of comparing the template image with the blade images; astep of selecting an image from the blade images based on a result ofthe image comparison in the comparing step, and a step of extracting adifference between the template image and the image selected in theselecting step. The image processing method also includes: a step ofextracting a reference image, which is distant by a predetermined numberof frames from the template image, from the blade images; a step ofextracting a difference between the template image and the referenceimage; and a step of detecting a blade region based on the differenceextracted in the difference extracting step.

While the embodiments of the invention have been described in detailwith reference to the accompanying drawings, the specific configurationis not limited to the above-described embodiments but a design changeand the like within the scope without departing from the subject matterof the invention are also included.

What is claimed is:
 1. An endoscope, comprising: an imaging elementdisposed at a distal end of the endoscope that obtains images through astereo optical adapter; a template image extracting unit comprisinghardware that extracts a template image from blade images obtained bythe imaging element by capturing blades periodically arrayed in a jetengine; an image comparing unit comprising hardware that compares thetemplate image with the blade images; an image selecting unit comprisinghardware that selects an image from the blade images based on a resultof the image comparison of the image comparing unit; a first differenceextracting unit comprising hardware that extracts a plurality ofdifferences between the template image and the image selected by theimage selecting unit; a measurement unit comprising hardware thatexecutes measurement based on an image corresponding to the position ofone of the plurality of differences extracted by the first differenceextracting unit; and an input unit comprising hardware to which aninstruction by a user to select one of the plurality of differencesextracted by the first difference extracting unit is input; wherein themeasurement unit executes, based on an image corresponding to theposition of the difference indicated by the instruction input by theuser to the input unit, stereo measurement using environmental data forcorrecting optical distortion of the stereo optical adapter to display aresult of the measurement based on the environmental data.
 2. Theendoscope according to claim 1, further comprising: a display unit thatdisplays information on the image selected by the image selecting unit,wherein the information indicates at least one of the differencesextracted by the first difference extracting unit.
 3. The endoscopeaccording to claim 1, wherein each of the blade images includes aplurality of subject images regarding the same subject, and themeasurement unit sets a measurement point at the position, whichcorresponds to the difference indicated by the instruction input by theuser, on one subject image included in the image selected by the imageselecting unit, calculates a matching point corresponding to themeasurement point on another subject image included in the imageselected by the image selecting unit, and executes measurement based onthe measurement point and the matching point.
 4. The endoscope accordingto claim 1, further comprising: a reference image extracting unit thatextracts a reference image, which is distant by a predetermined numberof frames from the template image, from the blade images; a seconddifference extracting unit that extracts a difference between thetemplate image and the reference image; and a detecting unit thatdetects a blade region based on the difference extracted by the seconddifference extracting unit.
 5. The endoscope according to claim 4,further comprising: a display unit that displays information on theimage selected by the image selecting unit, wherein the informationindicates at least one of the differences extracted by the firstdifference extracting unit and the blade region.
 6. The endoscopeaccording to claim 5, further comprising: a defect position determiningunit that determines whether or not the difference indicated by theinstruction input by the user, as a defect, is within the blade region,wherein the information indicates the defect determined to be within theblade region by the defect position determining unit.